1. Technical Field
The present invention relates to the optometry and optician fields, and more particularly to digital image processing and displaying apparatus for use by optometrists, opticians, eyeglass vendors, and related parties.
2. Background Art
When a person requires a new pair of eyeglasses, he or she usually visits an optometrist for an eye examination. Normally, an optician is also on hand to assist the person in selecting and fitting the new eyeglasses. After the eye exam, the person must select a frame for his or her eyeglasses. As many of us know, this process involves a review of many different frames made physically available to the person by the optician.
This process can be tedious, because: (1) the person may have to wait before being helped by an optician; (2) the person must choose from a large number of frame styles and manufacturers, usually requiring the assistance of the optician; (3) the person must physically try-on each of the frames selected for consideration; and (4) the person must preview each tried-on frame in a mirror without his or her prescription lenses, which can be difficult for some and inconvenient for most.
Also, when viewing frames on the face, the person does not see the optical effects of the prescription lenses to be placed in the frames. Thus, the person must select a frame without assessing this important aesthetic factor.
The above-described process requires the optician to display a large physical inventory of frames. This requirement is expensive, and inevitably limits the number and variety of frames that an optician can make available to the customer.
Once a particular frame is selected, the optician must then obtain certain measurements needed to produce the eyeglasses (i.e., the position of the customer""s eyes with respect to the frame). These measurements include the distance between the pupils, the ocular centers, the monocular pupillary distances, and the segment heights. Such measurements are critical for the proper manufacture of the eyeglasses for the selected frames. If such measurements are not properly taken, the resulting eyeglasses become useless to the customer, and time and money are wasted. Such measurements are usually taken manually with a ruler or with the aid of various measurement devices. All such methods require skill and great care to achieve acceptable accuracy. In practice, such accuracy is not always attained.
Different approaches have been proposed to overcome the various drawbacks mentioned above with respect to frame selection and eye/frame measurement. One particularly interesting approach is to simulate the frames (or eyewear) in a computer and digitally superimpose such eyewear on a digital image of the customer. Systems to carry out such an approach are proposed in the following patents: U.S. Pat. No. 5,280,570 to Jordan; U.S. Pat. No. 4,852,184 to Tamura et al.; U.S. Pat. No. 4,845,641 to Ninomiya et al.; and U.S. Pat. No. 4,730,260 to Mori et al. These computer-based systems have not gained widespread acceptance in the eyewear industry and by the general public.
One reason for this lack of acceptance is that such systems may not be presenting a realistic composite image of the customer wearing a selected frame. For instance, the systems proposed in U.S. Pat. No. 5,280,570 to Jordan, U.S. Pat. No. 4,852,184 to Tamura et al., U.S. Pat. No. 4,845,641 to Ninomiya et al., and U.S. Pat. No. 4,730,260 to Mori et al., do not display the temple portions of the frames on the face. The absence of these temple portions is immediately apparent to the customer, and adversely affects the credibility of the system as perceived by the customer. As a result, the customer is not likely to rely on such a system for selecting frames.
U.S. Pat. No. 4,539,585 to Spackova et al. proposes merging a frame having a temple portion with a face image, using separately displayed images of the frame and face and combining them with a half mirror. A digital processor embodiment is also proposed, however, it is not explained how the frame and face images are to be merged in the digital processor.
Further, as to the realism issue, the systems proposed in U.S. Pat. No. 5,280,570 to Jordan, U.S. Pat. No. 4,852,184 to Tamura et al., U.S. Pat. No. 4,845,641 to Ninomiya et al., and U.S. Pat. No. 4,539,585 to Spackova et al., do not address the problem of superimposing transparent and semi-transparent frames and frame parts (such as the nosepad) on a face image. Transparent and semi-transparent frames (or parts thereof) pickup background color when the frame image is created. This background color is very noticeable in the frame (or frame part) when viewing the composite image, and thus conveys a fake and not very convincing appearance. U.S. Pat. No. 4,730,260 to Mori et al. recognizes this problem, however, it fails to describe any real solution.
Another possible reason why the above-mentioned patented systems have not gained widespread acceptance is that such systems do not provide automatic measurement of the eye/frame parameters necessary to manufacture the lenses for the selected frame. U.S. Pat. No. 5,617,155 to Ducarouge et al. proposes a system to automatically obtain the eye/frame measurements by way of pupil and frame detection. However, in Ducarouge et al., the customer must wear a selected frame before his or her image is acquired by the system. Thus, the customer must choose his or her frame in the conventional manner before the system can be used, and the optician must carry a full complement of frames as is conventionally done. Similarly, in U.S. Pat. No. 5,592,248 to Norton et al., the customer must wear a selected frame before his or her image is acquired and measurements are made.
In U.S. Pat. No. 5,280,570 to Jordan it is mentioned that the shape of the customer""s face is an important factor in selecting frames. However, in Jordan, the customer or optician must first determine the shape of the customer""s face and then manually enter this parameter into the system. Such a process may be inconvenient and may yield an incorrect face shape determination. For example, it may be difficult for an inexperienced optician to distinguish between oblong and oval faces.
U.S. Pat. No. 5,576,778 to Fujie et al. proposes a system which acquires the face image of the customer and, through digital image detection, automatically determines the face shape and other facial features of the customer. The system then assists the customer in selecting frames that are best suited for his or her facial features. However, the Fujie et al. system does not simulate and superimpose eyeglass frames on the face image. Thus, the optician must carry a full complement of frames as is conventionally done.
It is therefore an object of the present invention to provide apparatus and methods that overcome the problems, drawbacks and limitations of the prior art.
It is another object of the present invention to provide apparatus and methods that aid the optician""s customer in selecting a suitable frame for his or her eyeglasses.
It is a further object of the present invention to provide apparatus and methods that substantially eliminate the tedium normally associated with selecting eyeglass frames.
It is yet another object of the present invention to substantially reduce the physical inventory of frames that an optician must carry.
It is yet a further object of the present invention to provide apparatus and methods that simulate eyeglass frames and digitally superimpose such frames on a digital image of the customer, to create a composite digital image.
It is still another object of the present invention to provide a realistic composite digital image of the eyeglass frames being worn by the customer.
It is still a further object of the present invention to provide apparatus and methods that simulate the complete eyeglass frame, including the temple portions.
It is yet still another object of the present invention to simulate transparent and semi-transparent eyeglass frames and frame parts.
It is yet still a further object of the present invention to conveniently provide a wide variety of eyeglass frames to the customer for consideration. it is yet still a further object of the present invention to provide apparatus and methods that automatically detect the irises and pupil centers in a digital face image of a person.
It is yet still a further object of the present invention to provide apparatus and methods that automatically detect the shape of a person""s face from a digital image of the person""s face.
It is yet still a further object of the present invention to provide apparatus and methods that automatically measure the eye/frame parameters needed to produce eyeglasses.
It is yet still a further object of the present invention to provide apparatus and methods that simulate contact lenses on the eyes of a person, in a digital image of the person.
It is yet still a further object of the present invention to provide an interactive eyewear selection system that is easy to use and is otherwise xe2x80x9cuser friendly.xe2x80x9d
It is yet still a further object of the present invention to provide a structure for an eyeglass frame image and to provide methods for creating such a structure.
These and other objects are attained in accordance with the present invention wherein there is provided an interactive eyewear selection system, comprising: means for acquiring a digital image of a person""s face, including the temple areas of the face; means for storing digital eyeglass frame images, said storage means containing a plurality of eyeglass frame images; means for selecting an eyeglass frame image from said storage means, said image containing frame temple portions; means for superimposing the selected frame image on the image of the person""s face, in a composite image, such that the frame appears to be worn naturally on the person""s face and the frame temple portions are shown at the temple areas of the face; and means for displaying the digital image of the person""s face and for displaying the composite image.
In accordance with the present invention there is provided a method of combining an eyeglass frame image with a face image. The method comprises the steps of: (a) obtaining a digital image of a person""s face; (b) selecting a digital eyeglass frame image, the frame image having left and right temple segments; and (c) digitally combining the frame image with the face image to produce a composite image that portrays an eyeglass frame superimposed on a face, the eyeglass frame portrayed in the composite image having left and right temple segments that correspond the left and right temple segments of the frame image.
In accordance with another aspect of the present invention, there is provided a method of detecting the irises and pupil centers of a person, in a digital image that includes the face of the person. The method comprises the steps of: (a) providing a gray level image that encompasses the eyes of the person, one eye being in a left area of the image and the other eye being in a right area of the image; (b) converting the gray level image to a binary image based on a dynamic threshold T; (c) searching the left area of the binary image for black objects and calculating the size and centroid coordinates of each black object found; (d) searching the right area of the binary image for black objects and calculating the size and centroid coordinates of each black object found; and (e) selecting a black object as a candidate for the iris in each of the left and right areas of the binary image, in accordance with the following selection criteriaxe2x80x94(i) the black object selected in the left area is approximately the same size as the black object selected in the right area, and (ii) the black object selected in the left area is approximately at the same level as the black object selected in the right area.
In accordance with another aspect of the present invention, there is provided a method of detecting the shape of a person""s face in a digital image that includes the face of the person. The method comprises the steps of: (a) determining the contour of the person""s face in the digital image; (b) converting the person""s face contour into the frequency domain by a Fourier transformation of a space domain function representing the contour; (c) providing a plurality of standard face contours represented in the frequency domain, each of the standard face contours having a name associated therewith; (d) comparing the frequency domain representation of the person""s face contour with each of the plurality of standard face contours, to determine the best match; and (e) assigning the name of the standard contour that was determined to be the best match in step (d), to the person""s face contour.
In accordance with another aspect of the present invention, there is provided a method of determining the frame parametersxe2x80x94monocular PDs, ocular centers, and seg-heights, automatically from a digital image of a person""s face and from a digital image of a selected eyeglass frame.
In accordance with another aspect of the present invention, there is provided a method of pre-processing digital images of eyeglass frames for storage in a frames database, and a particular data structure for the frame image. The frame processing method includes pre-processing for opaque, semi-transparent and transparent frames, and for the nosepad portion of the frame. Size scaling factors for the rims and bridge of the frames are calculated so that only one size need be stored in the frames database. Different size frames of a particular style are created digitally, based on the calculated scaling factors.
In accordance with a further aspect of the present invention, there is provided a method of selecting eyeglass frames in the interactive eyewear system by barcode scanning or manually entering an SKU It into the system. The frames database is indexed by SKU Its.
In accordance with yet another aspect of the present invention, there is provided a method of digitally superimposing color contact lenses on a person""s eyes, in a digital color image of the person""s face.